Install DB10gR2 on RHEL5
Installation requirements:

1. Installing required packages

The minimum required packages given here are sufficient only if Linux has been installed without customizing the package selection. This list is based upon a "default-RPMs" installation of RHEL AS/ES 5. Additional RPMs may be needed if a "less-than-default-RPMs" installation of RHEL AS/ES 5 is performed. For more information, please refer to Note 376183.1 "Defining a "default RPMs" installation of the RHEL OS".

The version numbers for these packages given below are the minimum version required. If you have the packages with higher versions than the versions mentioned in below example, you can install the higher version packages.

(i) gcc-c++-4.1.1-52.el5.i386.rpm and all its dependent packages. The dependent packages are
gcc-4.1.1-52.el5.i386.rpm
libstdc++-devel-4.1.1-52.el5.i386.rpm
glibc-devel-2.5-12.i386.rpm
glibc-headers-2.5-12.i386.rpm
libgomp-4.1.1-52.el5.i386.rpm
(ii) libXp-1.0.0-8.i386.rpm
(iii) compat-libstdc++-296-2.96-138.i386.rpm
(iv) compat-libstdc++-33-3.2.3-61.i386.rpm

Install all these packages from the directory Server in CD#2 of RHEL 5 by the single command.

# rpm -ivh gcc-c++-4.1.1-52.el5.i386.rpm gcc-4.1.1-52.el5.i386.rpm 
libstdc++-devel-4.1.1-52.el5.i386.rpm glibc-devel-2.5-12.i386.rpm 
glibc-headers-2.5-12.i386.rpm libgomp-4.1.1-52.el5.i386.rpm 
libXp-1.0.0-8.i386.rpm compat-libstdc++-296-2.96-138.i386.rpm 
compat-libstdc++-33-3.2.3-61.i386.rpm

(v) compat-db-4.2.52-5.1.i386.rpm
This package is available in the directory Server in CD#3 of RHEL 5. Install this package by the command

# rpm -ivh compat-db-4.2.52-5.1.i386.rpm 

2. Set the kernel parameters

Add the following the lines in the file /etc/sysctl.conf

kernel.shmmni = 4096 
kernel.sem = 250 32000 100 128 
fs.file-max = 65536 
net.ipv4.ip_local_port_range = 1024 65000 
net.core.rmem_default = 1048576 
net.core.rmem_max = 1048576 
net.core.wmem_default = 262144 
net.core.wmem_max = 262144 

To place these changes into effect, execute the command
# sysctl -p 

3. Create Oracle user and groups

# groupadd oinstall 
# groupadd dba 
# useradd -g oinstall -G dba oracle 
# passwd oracle 

4. Create the required directories for Oracle Base and change the ownership

For example 
# mkdir -p /u01/app/oracle 
# chown -R oracle:oinstall /u01/app/oracle 

5. Set the session limits for Oracle user

Add the following lines to the /etc/security/limits.conf file
oracle soft nproc 2047 
oracle hard nproc 16384 
oracle soft nofile 1024 
oracle hard nofile 65536 

Add the following line in the /etc/pam.d/login file
session required pam_limits.so 

Add the following lines to the /etc/profile.
if [ $USER = "oracle" ]; then 
if [ $SHELL = "/bin/ksh" ]; then 
ulimit -p 16384 
ulimit -n 65536 
else 
ulimit -u 16384 -n 65536 
fi 
fi 

6. Configure the Oracle user's environment

Configuring the oracle User's Environment, using the appropriate section of "Oracle® Database Quick Installation Guide 10g Release 2 (10.2) for Linux x86". Note that setting $ORACLE_BASE (not $ORACLE_HOME) in the environment will ease a couple of entries in the Oracle Universal Installer.
For example, if oracle user is using bash shell, and has the home directory as /home/oracle,
The following lines to be added to /home/oracle/.bash_profile
export ORACLE_BASE=/u01/app/oracle
export ORACLE_SID=orcl
unset ORACLE_HOME

7. Invoking OUI

Log in as Oracle user and start installation by any one of the following methods. This is required because Oracle 10.2 expects the OS to be RHEL 3 or RHEL 4.

Method 1:
$ ./runInstaller -ignoreSysPrereqs

Method 2:

Copy the oraparam.ini from the installation source install/oraparam.ini to /tmp directory.
Edit the file /tmp/oraparam.ini

Change the following section :
Before
[Certified Versions]
Linux=redhat-3,SuSE-9,redhat-4,UnitedLinux-1.0,asianux-1,asianux-2
After
[Certified Versions]
Linux=redhat-3,SuSE-9,redhat-4,UnitedLinux-1.0,asianux-1,asianux-2,redhat-5

Now invoke the runInstaller as follows : 
$ ./runInstaller -paramFile /tmp/oraparam.ini